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ABSTRACT 

Using the capability of the computer, the authors 
have designed an instructional program that emphasizes students' 
problem solving skills instead of their computational skills and that 
allows the collection of a large and detailed data base. This report 
describes the procedures used to (1) identify structural variables 
that affect students' performance on arithmetic word problems 
presented at a computer terminal, (2) identify variables for 
structuring a computer-based problem-solving curriculum, and (3) 
assess the usefulness of the identified variables as predictors of 
student performance on the newly structural curriculum. Fourth-, 
fifth- and sixth-grade students who were from one to three years 
below average in arithmetic computation skills were used as subjects 
for developing and testing the program and variable predictprs. 
Approximately two-thirds were black students from an economically 
depressed area and the remainder came from schools for the deaf. 
Results of several regression analyses revealed that it is possible 
to account i.or a substantial portion of the variability in student 
responses using significant correlation between the deaf and hearing 
students on a rank-order of the problem difficulty level. (JP) 
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Introduction 

A central theme of mathematics instruction is to help students 
develop problem-solving skills that generalize beyond the tasks of 
elementary-level mathematics curricuJums. Instruction in solving 
arithmetic word problems is one method of teaching problem^solving 
skills. Word problems are difficult for most students, and despite 
intense interest and investigation, much remains to be learned about 
the sources of problem difficulty. Using the capabilities of the 
computer, we have designed an intDcructional program that emphasizes 
students' problem-solving skills instead of their computacional skills, 
and that allows the colleccion of a Large and de^zalled data base. 

The study reported here has three purposes: (a) co identify 
structural variables that affect performance of students on arithmetic 
word problems presented at a computer terminal, (b) to use the 
identified variables to structure a computer-based problem-solving 
curriculum, and (c) to assess the usefulness of the identified 



(1) This research was supported by Office of Education Grant OEG- 
0-70-4797 (607) and NSF Basic Research Grant GJ-443X, 



variables as predlccors of sijjdent perfcrmance on the newly structured 
curriculum- 

The study was conducted in two phases. During the first phase 
700 arithmetic vord prohl3ms were written and edited. Predicting 
problem difficulty on the basis of results from pilot studies that used 
multiple linear-regression models, we structured the problem-solving 
curriculum by ordering problems from the least difficult to the most 
difficult. During che second phase, students who were enrolled in a 
computer-assisted instruction (CAI) arithmetic program given by the 
Institute for Mathematical Studies in the Social Sciences (IMSSS) of 
Stanford University were also given problem-solving (FS) lessons. The 
student population was drawn from several schools for the deaf and 
from a school in an economically disad^ ntaged area. Using the 
perforinance data collected for these students to reanalyze problem 
difficulty, we revised the sez of structural variables and made new 
predictions for the problem set, whir-h permitted a restructuring of the 
curriculum . 

Description of_ the Problem-solvi ng Course 

The PS course is designed to give students practice in solving 
arithmetic word problems. Tutorial functions of the course are limited 
to giving general and, in some cases, specific hints, The emphasis of 
the course is on methods of solution; the student constructs a well- 
formed algebraic expression, but it is the computer that carries out 
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the computations. Thus, the studant learns a set of simple commands 
that tell the computer which computations to carry out, Although the 
text of the problems is stored by the computer, the numbers used in 
each problem are generated for each presentation. As a result, 
students who see the same problem starement have different numbers to 
work with. 

In solving problems, the student is free to experiment with the 
computer calculator made available to him. The computer evaluates 
his response only when he instructs it to do so. The student is given 
three chances to respond correctly to a problem. After the third 
error, the computer types the correct numeri::al answer to the problem, 
and then types the problem again with newly generated numbers. The 
student may skip a problem at any time, and, as a result, is not 
obliged to solve every problem. Examples of problem output with 
student input are shown m Figure K Student inpuc is underlined. The 
student uses the equal sign (=) to request evaluation 

Insert Figure t about here 

The student constructing a solution in the PS course is also 
free to use any combination of steps. The computer calculates the 
correct answer from a stored solution string using the numbers 
generated for the problem presentation, and compares the result with 
the student's answer. 



PROBLEM no3 



TOM HAS 63 CARS. ME GIVES ^ TO MAX. HOW MANY CARS HOES 
TOW HAVE NOW? 

A « 63 

B « ^ 



TRY SUBTRACTING. 

C = &9 
WELL DONC 



PrlOBLEM 1106 

MR. BROWN HAS i9830 IN ONE BANK AND HF HAS %b90 IN ANOTHER BANK, 
TOO. HOW MUCH MONEY DOES HE HAVE IN THE TWO S(\fiK$? 

A = 9830 

B = *590 



»AXB= 

NOT GiUITE* TRY AGAIN* 

»A-«-B= 

D = I04ii0 
BETTER 



ERLC 



PROBLEM II Uy 

JUAN HAS PET FISH, 3 PET CATS* AND 7 PET RABBITS- 
HOW MANY PETS DOES HE HAVE IN ALL? 

A = yy 

B 3 
C V 



♦A + B 

D = 1U2 



E ^ 109 
GROOVY 



HHOBLEM n 1 tt 

DANIEL HAD 91 ROCKS AND HE GAVE THEM TO 5 CHILDREN. 
HOW MANY ROCKS DID EACH CHILD GET? 

A ^ yi 

B ^ b 



C = 18.200 
GREAT 

Fig. 1. Sample output from the PS course- 



Pilot Study I 

Data from studies by Loftus and Suppes (]972) and Suppes , 
Loftus, and Jerman (1969) were used to calculate a linear rep. -ssion 
model for performance on arithmetic word problem solving. These 
studies were based on 100 word problems of appropriate difficulty for 
sixth-grade students- Sixty-five of ths 100 problems A^ere completed by 
all subjects in both studies and were used in the present analysis. 

Subjeccs in the Suppes, Loftus, and Jerman scudy (Group 1) were 
27 students from an accelerated mathematics class composed of fifth 
graders from several upper middle-class elemencary schools- Subjects 
in the Loftus and Suppes study (Group 2) were 16 sixth graders from two 
schools in a culturally disadvantaged area. The two groups performed 
quite differently. The mean percentage correct on the set of 65 
problems was 85.0 (SD = 17.6) for Group 1 and 56.7 (SD = 28,:>) for 
Group 2. In the present analyses, daca for both groups were pooled. 
The mean percentage correct for the Pooled Group was 74^5 (SD = 20 0) . 

Variables Characterizing^ Problem Dif r iculty 

The variables chosen co characterize problem difficulty are 
listed and defined in Table 1. These variables describe aspects of 
arithmetic word problems. Because this study emphasizes developing a 
curriculum, the variables are not exhaustive; instead they encompass 
major features of problem structure- A more detailed set of variables 
is presented in Jerman (1971). Most of the variables fall into one of 



two groups: those that describe a standard solution algorithm for the 
problem, and those tliat describe the textual statement of the problem. 
A few variables depend for their definition on mathematical aspects of 
the problem that cannot be unambiguously identified in a solution 
algorithm. 

Insert Table 1 about here 

A standard solution algorithm was written for each problem. 
For most problems such an algorithm could be specified unambiguously. 
Where two or more different , but correct , algorithms could be 
constructed, the choice depended on (a) the method presented for 
^solving the problem in standard elementary-level mathematics texts, (b) 
the intuitive judgment of the authors based on their experience with 
children's problem-solving behavior, and (c) the variable values 
assigned to the algorithm. When two algorithms were judged 'natural', 
using the criteria described in (a) and (b) , the algorithm that gave a 
minimal sum of the variable values was chosen « 

Variables that desci ibe characteristics of the solution 
algorithm are OPERS, STEPS, ORDER, ADD, SUB, >nJL, and DIV. The 
variables OPERS, ORDER, and STEPS are not independent; the value of 
STEPS places an upper limit on the possible values of OPERS and ORDER. 

Variables that describe the textual statement of the problem 
are LENGT and VCLl'^l. A simple measure of verbal complexity, number of 
words in the problem statement (LENGT) , was chosen for this study . 



TABLE 1 



Definition of Variables Used for Pilot Study 



Variable Name Range Definition 

X OPERS 1-3 Minimum number of different arithmetic 

1 operations required to reach a solution. 

X STEPS 1-7 Minimum number of binary operations 

2 required tio obtain an answer. 

X LENGT 7-51 Number of words \n the problem Each 

3 number symbol coun':s as one word ^ 

X CONVR 0,1 Problem is said to have a conversion 

4 (coded 1) if conversion of units is 

required and the equivalent units are 
not presented in the problem statements 

X VCLUE 0,1 Problem has a verbal clus (coded 0) if 

5 (a) there is a clue for each required 

operation, and (b) i£ the clue word 
(or phrase) is one of the following: 

for +, added, altogether, gained; 
for -, how much (less, more and 

synonyms) ; 
for X, each; 
for / , average . 

X ORDER 0,1 Order is the same (coded 0) if the 

6 numbers in the problem aie presented 

in the same order as they occur in 
the coded solution string. 

X FORMU 0,1 Solution of the problem requires 

7 knowledge of a formula not included 

in the problem presentation (coded 1). 

X AVERG 0,1 The word average is m che problem 

8 statement, and tlie student must compute 

an average or usa an average to solve 
the problem (coded 1). 

0,1 Solution requires an addition. 

0,1 Solution requires a subtraction. 

0,1 Solution requires a multiplication. 

0,1 Solution requires a division. 



X 


ADD 


9 




X 


SUB 


10 




X 


MUL 


n 




X 


DIV 


12 
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Although Loftus and Suj^pt'S (197.?) ro^ortcu Llic adv/inL.'if^.os of jM(:lud:inf> 
a measure that ch.'?racterizc>y tlie structural coinpiexity of sentences, 
difficulties of coding this mensure precluded its use in tliis study. 
The variable VCLl']",, v:hic'! .Ind.Lt^itcs the presence of a verbal clue in 
the problem statement, depends on hoth verbal and mathematical 
properties of the problem. 

The variables CONVR and FORMU describe problems that require, 
respectively, conversion of units and knowledge of a formula. 
Frequently problems of these types call for the student to use a 
number, a conversion factor, or otht- constant not presented in the 
problem statement. The same is true of the variable AVERG, which also 
requires the presence of the word 'average' in the problem text. 

The Regression Model 

A stepwise, multiple regression-analysis program (Dixon, 1970), 
adapted for the Institute's PDP-10 computer system, calculated 
regression coefficients, standard errors of estimate, multiple 
correlation coefficients (R) , and the square of the multiple 
correlation coefficients for the 65 problems completed by the Pooled 
Group. Proportion correct was the dependent variable in these 
reprosr; ' 0115^ . Suppes, Loftus, and Jerman (1969) describe the regression 
model in detail. 
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The rs>;res8ion equacion was 

iff^ it /<* 

z = -1.80 + .18X + .02X + .OIX + . j?A + .34.\ 
i 11 x2 IJ i4 15 

+ .04X + 83X + .05X - .08X + . 26X , 

16 l8 19 111 112 

*n < .05; 
^-'P ^ .005; 

with a mitltlple R ot .81, a ,st;a, J.rd cvn>r ot estimate of .36, and 
2 

an R of .66. The order In which variahles were entered Into the 

stepwise regression is presented In Table 2. Six variables, OPERS, 

VCLUE, DJV, LENGT, FORMU, and COiNVR, accounted for 64 percent of the 

dependent variable variance. The variables FOR.MU and CONVR 

characterized few of the problems; the mean coding for FOR'IU was *03 

and for CONVR was .08. These two variables wets combined into a single 

variable, CONFO (X ), uliich v/as coded as \ if the problem solution 

4 

required either a conversion or knowledge of a formula ^ 



Inserc Table 2 about hete 



The reft>:ession equa-iion^ usin^ the five variables, OPERS, 
LENGT, CCNFO, VCLUE, and DIV, was 

* * X * 

(1) z = -K79 + .23X + .02X + .46X + .27X + .34a 

i il 13 14 15 i12 

*p < .005; 

with a multiple R of .78, a standard error of estimate of .37, and 



ERIC 



TABLE 2 

Order of Introduction of the Variables 
In the Regression (Pilot Study Data) 



Variable Multiple R Standard error of 

estimate 



X OPERS .66 .43 

1 

X VCLUE .70 .41 

5 

X DIV .73 .40 

12 

X LENGT .76 .38 

3 

X FORJW .79 .36 

7 

X CONVR .80 .36 

8 

X ADD .81 .36 

9 

X AVERG .81 .36 

fo 



X MUL 
1 1 



.81 .36 



X STEPS .81 .36 

2 

X ORDER .81 .36 

6 



O 10 
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an R of .60. Table 3 presents the regression coel f iclents , T 
values, and partial correlation coefficients computed for each of the 
five independent variables. 

Insert Table 3 about here 



Construction of the Curriculum 

Equation 1 was used to predict the probability correct for each 
of the 700 problems written for the PS course. The probabilities 
obtained ranged from ,95 to .07. Using the calculated probabilities, 
we constructed the curriculum by ordering the problems from easiest to 
hardest. In addition to the 700 ordered problems, 39 introductory 
problems were written to instruct students on interacting with the 
program. Fourteen nonnumerical problems taught the students to find 
characters on the teletypewriter keyboard, to ask for a hint, and to 
request an evaluation of an answer. Twenty-five numerical problems 
illustrated different problem types and ranged in predicted difficulty 
level from .79 to .95. 
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TABLE 3 

Regression Coefficients, Standard Errors of Regression 
Coefficients and Computed T Values (Pilot Study Data) 



Variable Regression SE Computed 

coefficient T value 



X OPERS 



X LENGT 
3 



X DIV 
12 



,233 .077 3.026 

.017 .005 3.400 



X CONFO .459 .158 2.905 

4 

X VCLUE .272 .099 2.747 

5 



,337 .107 3.149 
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Subjects 

The experimental subjects were fourth, fifth, and sixth graders 
enrolled in the IMSSS arithmetic CAl course. Approximately two-thirds 
of the students came from a primarily black California elementary 
school and the remainder came from schools for the deaf in several 
parts of the country, 

The black students were from an economically depressed area in 
Santa Clara County, California, w^ere the school discrict comprises 5 
percent of the total county school population. Of the entire 
population of county welfare families, 35 percent live within the 
school discrict. Students in Grades 4-6 are, on che average, from one 
to three years below grade level in arithmetic computation skills. 

The majority of the deaf students were enrolled in residential 
schools for the deaf in several parts of the country. The degree of 
hearing loss among the students (at least 60 decibels in the better 
ear) was that adopted for admission standards by che participating 
schools. Such deaf students are, on the average, from two to three 
years below grade level in arithmetic computation skills. 

Each student took arithmetic lessons at a teletypewriter 
terminal connected to the IMSSS PDF- 10 computer system by telephone 
lines, A student became eligible for the PS course when his average 
grade placement on the CAI arithmetic program reached 4.0. Thereafter, 
if his teacher chose to enroll him in PS, he received a PS session 
every fifth day. Thus each student started the course at a different 
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time of year and proceeded -ac his own pace throa^^h clie curriculum. Of 
approximately 300 students who received some portion of the course, 120 
completed the introductory problems. Approximately 50 students in this 
group went on to complete the first 100 ordered problems. The data 
reported here are iror 125 problems, the 25 numerical introductory 
problems, and the first 100 problems of the ordered set From 51 to 309 
responses were recorded for individual problems. 



Results 

Although the students in this jtudy came from two very 
different disadvantaged populations, thei?' performance was similar in 
this setting. Mean values for six performance measures for deaf and 
hearing students are presented m Table 4 Two measures are for 
responses chat were correct cn ttie first, try; the number of steps used 
to reach a ct^rrect solution, and the time in minutes from the 
completion of the problem presentation at the terminal to the request 
for evaluation by the student, Th^ measures for incorrect answers 
record the time and number of steps used by the student to complete the 
problem when his first response was incorrect.. Recall that the student 
was given a maximum of three opportunities to have his answer 
evaluated. Also included are the proportion of correct responses and 
the proportion of problems for which a hint was requested. The deaf 
and hearing students did not differ on any of these performance 
measures. In addition, there is a significant correlation between the 
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rank-order of problems for the two groups (Kendall's rho = .5^1, 
p < •001), indicating that both groups found the same problems easy or 
hard . 



Insert Table 4 about here 

Our finding of similarity between two disadvantaged populations 
is significant. The students whose responses were examined do n*. t 
represent random samples of the two disadvantaged groups, since 
eligibility for the course depended on a minimal performance level, and 
all the students did not complete the same number of problems. 
Presumably, those students least able to cope with the course dropped 
out earliest. Nevertheless, the types of handicaps characterizing the 
two groups do not seem to produce differential performance in this 
setting. For all further discussion of experimental results, data for 
the two groups were pooled. 

The proportion correct for each problem was obtained, and the 
distribution of these proportions is shown in Figure 2. Although 
predicted probability correct for the 125 problems used in the analysis 
ranged from .79 to .95, the observed proportions ranged from .03 to 
* 94 . Moreover, a comparison of problem order for observed, and 
predicted proportions correct indicated that the rank of the observed 
values was random with respect to the previously established ranking 
(Kendall's rho = -.086). The proportion of correct responses for 70 
problems fell in the range .60 to .94. For all but 5 problems, the 
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Measure Mean 

Deaf Hearing 

Proportion correct .692 .706 

Number of steps for 1.170 1.129 

correct solution 

Latency for correct .442 .422 

solution (min.) 

Number of steps for 3.045 3.036 
incorrect solution 

Latency for incorrect 1.221 1.274 
solution (min.) 

Proportion of hints ,179 .141 

requested 
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observed proportion correct was lower than predicted. The mean 
difference between observed and predicted proportions was -.22, Thus, 
the pilot study overestimated student pfjrf ormance , but this is hardly 
surprising in view of the superior ability of some of the students 
whose response data constituted part of the pilot study. 

Insert Figure 2 abouL here 

There were 21 problems fir which the difference becween the 
predicted and observed proportions was greater than -.40, Ten of these 
were introductory problems and some poor performance could be accounted 
for by the unfamilarity of problem cypes selected for illustration. An 
examination of the remaining i i indicated that the range of the 
variable ORDER should be expanded and that more attention should be 
given to the length of words in problem statements, 

Two par t'^cuiar ly diificu'Lt problems wer<=, "What number divided 
by # gives //?" and "What tiaction or # is #V" (The is replaced by 

a program-generated number in presenting the problem,) One possible 
explanation for the difficulty or these problems is the terseness of 
the statement and the absence of a setting or ^story' , This suggested 
that a new variable be defined to distinguish between 'algebraic' and 
'story* problems o 
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Fig. 2. Frequency distribution of average proportion correct 
for 125 PS course problems • 
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A regression analysis for all 125 problems was recalculated, 
using observed proportion correct as the dependent variable and the 
same independent variables as in Equation (1). The re0;ression equation 
was 

z -1.80 + 1.19X - ,001X + K36X + .21X + .85X 
i i1 13 14 15 i12 

*p < .005; 
**p < .001; 

with a multiple R of *66y a standard error of estimate of .38, and an 

2 2 
R of .44, which was considerably lower than the R of .60 obtained 

with the pilot study data. 

Table 5 presents regression coefficients, T values, and partial 

correlation coefficients for each of the five independent variables* A 

comparison of these results with those in Table 3 shows the increased 

contribution of OPERS, CONFO, and DIV, and the decreased contribution 

of LENGT to predicted probability correct. 

Insert Table 5 about here 

The regression coefficient for LENGT was no longer 
significantly different from zero. This finding is surprising 
considering the language difficulties of deaf children, who constituted 
approximately one-third of the student group. Problem length in the 
pilot study ranged from 7 to 51 words (mean = 29.61, SD = 9.69) and 
in the PS curriculum from 7 to 39 words (mean =21.14, SD = 8.73). 
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TABLE 5 

Regression Coefficients, Standard Errors of Regression 
Coefficients, and Computed T Values (PS Course Data) 



Variable Regression SE Computed 

coei'ficient T value 



X OPERS 1.194 .176 6.767 

1 

X LENGT -.001 ,005 .184 

3 

X CONFO 1.362 .479 2.841 



X VCLUE .207 .072 2.860 

5 

X DIV .851 .164 5.195 

12 
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Thus, problems in rlu PS curri aiimii wi-:re ^'.eneraxly shorter. ]'owever, 
this difference does nou seem suf. riiienc to account for the 
independence of probability correct and problem length. The four 
remaining variables have significant regression coefficients, and, 
except for VCLUE, aca substantially larger than the corresponding 
coefficients from the pilot study anaiy.sis. 

Before continuing die aaalysis, the definition of several 
variables was sharpened, c:hers were divided into two variables, and 
several new variables were defined- The variables used for further 
analysis of the PS data are shown in Table 6. 

Inserc Table 6 about here 

Thirteen additional variables were defined r Three of these, 
MAXWD, MAXSN, and NLR-ISN, chaiaccerize Lhe problem statement in greater 
detail than LENGT- After an examination of the raw data, some specific 
variables (19-22) describing the structure of subtraction problems were 
defined. The variables SEQUE and POSIT attempt to account for the 
position of a problem in relation to neighboring prcblem types (SEOrE) 
and for the amount: of practice the student is likely to have had 
(POSIT) . 

The multiple regression analysis was repeated using 23 
variables • There were no problems exemplifying FORitU and AVRG1 , This 
analysis yielded a multiple R of c90, a standard error of estimate 
2 

of .24, and an R of .81. The order of variables entered in the 
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TABLE 6 



Definition of Vncicbles Used for Analysis of 1\S Course Data 
Variable Name Range Definition 



X OPERS* 1-2 Number of different arithmetic operations 

1 1-4 required to reach a solution, using the coded 

solution string- 

X STEPS^ Number of binary operations required to obtain 

2 1-9 an answer, using the coded solution string, 

X LENGT 7-79 Number of words in the problem. Each number 

3 symbol (it) counts as one word, 

X C0NV1 0,1 3'roblem is said to have a conversion (coded 1) 

4 if conversion of units is required and the 

equivalent units are not presented in the 
problem statement. 

X VCLUE* 0,1 Problem has a verbal clue (coded 0) if (a) 

5 operation is 4- and problem has word 'together' 

or 'altogether ' , or if (b) operation is ~ and 
problem has phrase 'have left' or 'were left', or 
if (c) operation is X and problem has word 'each', 

X ORDER* 0-2 The number of adjacent pairs of letters in the 

6 0-3 solution string that are not in alphabetical 

order . 

X FORl'lU 0,1 Solution of the problem requires knowledge of 

7 a formula not included in the problem 

presentation (coded 1). 

X AVRG1* 0,1 The word average is in the problem statement, 

8 and the student uust compute an average 

(coded 1 ) . 

X ADD 0,1 Solution requires an addition. 
9 



Note. — Range printed in brackets characterizes full 700-problem set. 
*Def inition different from that presented in Table 1 . 
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TABLE 6, cont. 



Variable Name Range Definition 

X SUB Otl Solution requires a subtraction. 
10 

X ^^UL 0,1 Solution requires a multiplication. 
11 

X DIV 0,1 Solution requires a division. 
12 

X MAXWD Length of longest word in problem. 

13 4-16 

X MAXSN 5-30 Number of words in longest sentence. 

14 6-37 

X NUMSN 1-4 Number of sentences. 
15 

X C0NV2 0,1 Problem requires a conversion of units and 

16 equivalent units are presented in the problem 

(coded 1). 

X ALGER 0,1 Problem statement is an algebraic statement, 

17 not a 'story* (coded 1). 

X CONST 0-1 The number of constants in the coded solution 

18 0-4 string. (Overlaps AVERG and CONVI.) 

19-22 Type oi subtraction problem (coded 1). 

X SUBT1 0,1 Type 1: Have a, take away b. How many left? 
19 

X SUBT2 0,1 Type 2: Have b. How many more do you need to 

20 make a? 

X SUBT3 0,1 Type 3: "b" + "c" = a. *'b" = b. Therefore ''c'**? 
21 

X SUBT4 0,1 Type 4: "a" - "c" - b. **a" = a. Therefore "c''«? 
22 

X POSIT 1-4 Position in problem set, problems // 1-25 coded 4, 

23 26-50 coded 3, 51-75 coded 2, 76-100 coded 1- 

X SEQUE 0,1 Coded 1 if solution string of preceding problem 

24 is exactly same as current problem. 

X AVRG2 0,1 The word 'average' is in the problem statement 

25 and student must use an average to solve the 

problem. 
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regrcBBionp the multiple U, and the stepwise Increase in R are 
presented in Table 7« 

Insert Table 7 about here 



2 

Seven variable's increased ]\ more than 1 percent. These were 
ORDER, OPKHS, ALCKH, ADD, SUBTl, DIV, and STEPS, Three of these 
variables contributed significantly to the prediction of probability 
correct for the pilot study > Of the newly defined variables, orly 
ALGER and SUBTl are included in this group. It is interesting that no 
variables characterizing word and sentence length contributed 
significantly to the regressions. Because SUBTl was highly correlated 
with SUB (r « .73), SUB was used ir place of SUBT1 in further 
analyses. The regression equation using the variables ORDER, OPERS, 
ALGER, ADD, SUB, DIV, and STEPS was 

z » -K76 + 1.10X + . 19X + .81X 
i il 12 i6 

ieit it it 

.35X - .18X + .35X + .A3X 
19 i10 i12 il7 

*p < .005; 
**p < .001 ; 

with a racltiple R of .85, a standard error of estimate of *27, 
2 

and an R of .73. Thus, nearly 75 percent of the variability in 
student response to 125 problems in the PS course was accounted for by 
seven structural variables. 
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TABLE 7 

Order of Introduction of the Variables 
in the Regression (PS Course Data) 

2 

Variable Multiple R Increase in R 

X ORDER -571 .326 

6 

X OPERS .734 .212 

1 

X ALGER .^^0 .070 

17 

X ADD .814 .054 

9 

X SUBT1 .846 .052 

19 

X DIV .857 .018 

12 

X STEPS .867 .018 

2 

X C0NV2 .873 .010 

16 

X POSIT .878 .009 

23 

X VCLUE .885 .012 

5 

X SEQUE .891 .009 

24 

X CONST .892 .002 

18 

X MAXSN .893 .001 

14 

X MAXWD .894 .001 

13 

X LENGT .895 .001 

3 

X NUMSN .895 .001 

15 

X SUBT4 .896 .000 

22 

X SUBT2 .896 .000 

20 

X MUL .896 .000 

11 

X SUBT3 .899 .006 

21 

X C0NV1 .900 .000 

4 

X AVRG2 .900 .000 

25 
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Suimnary 

We have shown that it is possible to account for a substantial 
portion of variability in student responses to arithmetic word problems 
using variables that describe struccurai features of the problems. 
However, the results obtained at this scage in our investigations are 
situation-dependent.. The greater variance in observed proportion 
correct compared with the variance in the predicted proportions, and 
the differing sets of variables conlributing significantly to the 
regressions come as no surprise. First, the population for this study 
differed from chat used in the pilot study. Second, it is clear that 
characteristics of the problem set, for example, the frequency of 
occurrence of exemplars for the range of values for each variable and 
the way variable values are combined in problem types, affect the 
weighting for each variable in the re^:ression analysis. Thus, 
differences were expected because different problem sets were used for 
the pilot study and the present study. 

In the light of rhese differences, the similarity in 
performance of the two disadvantaged groups gains in significance, and 
deserves further study. V/e believe we can increase the 
generalizability of our results by redesigning the basic problem set to 
exemplify in a balanced fashion the full range of variables found to 
account for problem difficulty » Given, however, the difficulty of 
making accurate predictions about problem-solving results, the 
correctness of this belief needs to be explicitly tested. 



26 



References 

Dixon, W. J. B>ID Biomedical Computer Programs. Los Angeles: 

University of California Press, 1970. 
Jerman, M. Instruction in problem solving and an analysis of 

structural variables that contribute to problem-solving difficulty* 

Technical Report No. 180. Stanford: Institute for Mathematical 

Studies in the Social Sciences, Stanford University, 1971. 
Loftus, E., & Suppes, P. Structural variables that determine 

problem-solving difficulty in computer-^assisted instruction. 

Journal of Educational Psychology , 1972, 63^, 531-542. 
Suppes, P., Loftus, E., & Jerman, M. Problem solving on a 

computer-based teletype. Educational Studies in Mathematics , 

1969, 2_, 1-15. 




27 



(Continued from inside front cover) 



ERIC 



165 L, J. Hubert. A form<ii niodeJ for ihe perceptual processinq of geometric configurations. Fcb-uary 19, 1971. (A statistical melfiod for 

investigating Ifie perceptuat confusions .imong cjeumetric configurations. J ournal of Mat hcrrmticai Psycfiology , 1972, 9, 389-403.} 

166 J. F. Juola, I. S. Fiscfiler, C. T, Wood, and R. C. Atkmson, Recognition time far information stored in long-term memory. (Perception and 

Psychophy&ics , 1971, 10, 8-1^.) 

167 R. L. Klatiky and R. C. Atkinson. Specialisation of Ific ccrebnl lu»mispliercs in scanning lor information in sfiofl-term memory. ( Perception 

and Psychopfiysics , 1971, 10, 335-338.) 

168 J. D. Fletcher and R. C. Atkinson. An evaluation of tfie Stjnfi.nri CAI progrii^ri in initial rending (grades K Ifirotigfi 3). Marcfi 12, 1971. 

(Evaluation of the Stanford CAt program in initial reading. Jou''nai of Educational Psychology, 1972, 63, 597-602.) 

169 J. F. Juola and R. C. Atkinson. Memory scanning for words versus categories. (J ounal of Verbal Learning and Verbal Behavior, 1971, 

10,522-527.) 

170 I, S, Fischler and J. F. Juold. Effects of repeated lei>ts on recognition time for information in long-term memory. ( Journal of Experimental 

Psychology , 1971, 91, 54-58.) 

171 P. Suppes. Semantics of context-free fragments of natural languages. March 30, 1971. (InK. J. J, Hlnlikka, J. M. £. Moravcsik, and 

P. Suppes (Eds.), Approaches to natural l anguage . Dordrecht: Reidel,1973. Pp. 221-242.) 

172 J. Friend. INSTRUCT coders' manual . Mayl,1971. 

173 R. C. Atkinson and R. M. Shiffrin. The control processes ol short-term memory. April 19, 1971. (The control of short-term memory. 

Scientific Americ cin, 1971, 224, 82-90.) 

174 P. Suppes. Computer-assisted instruction at Stanford. May 19, 1971. (In Mar and computer . Proceedings of international conference, 

Bordeaux, 1970. Base); Karger, 3 972. Pp. 298-330.} 

175 D. Jamison, J. D. Fletcfier, P. Suppes, and R. C. Atkinson. Cost and performance? of computer-assisted instruction for education of disadvantaged 

children. July, 1971. 

176 J. Offir. Some inatfiematical models of individual differences m learning and pc?rformance. June 28, 1971 . (Stochastic learning models with 

distribution of parameters. Journal of Matliematica l Psychology , 1972, 9(4), ) 

177 R. C. Atkinson and J. F. Juola. Factors influencing speed and accuracy of word recognition. August 1 2, 1971 . (In S. Kornblum (Ed.), 

Attention and performance iV. New York: Academic Press, 1973.) 

178 P. Suppes, A. Goldberg, G. Kanz, B. Searle, and C. Stauffer. Teacher's handbook for CAI courses. September 1, 1971. 

179 A. Goldberg. A generalized instructional system for elementary mathematical logic. October 11, 1971. 

180 M. Jerman. Instruction in problem solving and an analysis of structural variables that contribute to problem-solving difficulty. November 12, 

1971. (Individualized instruction in problem solving in elementary mtitheniatics. Journal for Research in Mathematics Education , 1973, 
4, 6-19.) 

181 P. Suppes. On the grammar and model-theoretic semantics of rihildren's noun phrases. November 29, 1971. 

182 G. Kreisel. Five notes on the application of proof theory to computer t-cience. December 10, 1971, 

183 J. M. Moloney. An Investigation of college student performance on a logic cirrriculum in a computer-assisted instruction setting. January 28, 

1972. 

184 J. E. Frieno, J. D. Fletcher, and R. C. Atkiitson. Student performance in computer-assisted instruction m programming. May 10, 1972. 

185 R. L. Smith, Jr. The syntax and'semantics of ERICA. June 14, 1972. 

186 A. Goldberg and P. Suppes. A computer-assisted instruction program for exercises on finding axioms. June 23, 1972. (Educational Studies 

in Mathematics , 1972, 4, 429-449.) 

187 R.C.Atkinson. Ingredients for a theory of instruction . June 26, 1972. ( American Psychologist , 1972, 27, 921-931.1 

188 J. D. 8onvillian and V. R. Charrow. Psycholingiii5tic implications of deafness: A review. July 14, 1972. 

189 P. Arable and S. A. 8oorman. Multidimensional scaling ol measures ol distance between partitions. July 2b, 1972, (Journal of Mathematical 

Psychology , 1973, 10, ) 

190 J. Ball and D. Jamison. Computer-assisted instruction for dispersed populations: System cost models. September 15, 197 2. ( Instructiona l 

Science , 1973, 1, 469-501.) 

191 W. R. Sanders and J. R. Ball. Logic documentation standard for the Institute for Mathematical Studies m the Social Sciences. October 4, 197?. 

192 M. T. Kane. Variability tr the proof behavior of college students in a CAI cour^^e in logic as a function of problem characteristics. October 6, 

1972. 

193 P. Suppes. Facts and fantasies of education, October 18, 1972. (In M. C. Wittrock (Ed.), Changing education : Alternatives from educationai 

research . Englewood Cliffs, N. J.: Prentice -Hall , 1973 . Pp. 6-45.) 

194 R. C. Atkinson and J. F. Juola. Search and decision processes in recognition memory. October 27, 1972. 

195 P. Suppes, R. Smith, and M. Leveille'. The French syntax and semantics of PHILIPPE, part 1: Noun phrases. November 3, 197 2. 

196 D. Jamfson, P. Suppes, and S. Weils. The effectiveness of alternative instructional methods: A survey. November , 1972. 

197 P. Suppes. A survey of cognition in handicapped children. December 29, 1972. 

198 8. Searte, P. Lorton, Jr., A. Goldberg, P. Suppes, N. Ledet, and C. Jones. Computer-assisted instruction program: Tennessee State 

Universfty. February 14, 1973. 

199 D. R. Levine. Computer-based analytic grading for Gernan grammar instruction. March 16, 1973. 

200 P. Suppes, J. D. Fletcher, M. ZanottI, P. V. Lorton, Jr., and 8. W. Searle. Evaluation of computer-assisted instruction In elementary 

mathematics lor hearing-impaired students. March 17, 1973. 

201 G. A. Huff. Geometry and formal linguistics. April 27, 1973. 

202 C. Jensema. Useful techniques for applying fatent trait mental-test theory. May 9, 1973, 

203 A. Goldberg. Computer-assisted instruction: The application of theorem-proving to adaptive response analysis. May 25, 1973. 

204 R. C. Atkinson, D. J. Herrmann, and K. T. Wescourl. Search processes in recognition memory. June 8, 1973. 

205 J. Van Campen. A computer-based introduction to the morphology of Old Church Slavonic. June 18, 1973. 

206 R. B. Kimball. Self-optimizing computer-assisted tutoring: Theory and practice. June 25, 1973. 

207 R. C. Atkinson, J. D. Fletcher, E. J. Lindsay, J. 0. Campbell, and A. 8arr. Computer^assisled instruction in initial reading. July 9, 1973. 

208 V. R. CharrowandJ. D. Fletcher. English as the second language of deaf students. July 20, 1973. 

209 J. A. Paulson. An evaluation of instructional strategies in a simple learning situation. July 30, 1973. 

'"210 N. Martin. Convergence properties of a class of probabilistic adaptive schemes called sequential reproductive plans. JuJy 31, 1973. 



(Continued from inside back cover) 

211 J. Friend. Computer-assisted instruction in programming: A curriculum description. July 31, 1973. 

212 S. A. Weyer. Fingerspelling by computer. August 17, 1973. 

213 B. W. Searle, P. LortOH/Jr., and P. Suppes. Structural variables affecting CAI performance on arithmetic word problems of disadvantaged 

and deaf students. September 4, 1973. 



o 

ERIC 



